Techniques for rendering complex scenes

ABSTRACT

Techniques that facilitate rendering of a scene of arbitrary complexity without having to load all the models involved in the scene simultaneously in computer memory. A representation of the inputs and outputs of the various models involved in the scene and the dependencies between the inputs and the outputs is provided and used to facilitate the rendering process. A tessellator may tessellate the models without having to load information for all the models concurrently into computer memory. The tessellated scene may then be rendered without having to load all the models in the scene concurrently into computer memory.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application incorporates by reference for all purposes theentire contents of the following applications:

(1) U.S. patent application Ser. No. 10/843,690, entitled STORINGINTRA-MODEL DEPENDENCY INFORMATION, filed concurrently with the presentapplication;

(2) U.S. patent application Ser. No. 10/843,688, entitled TECHNIQUES FORPROCESSING COMPLEX SCENES, filed concurrently with the presentapplication; and

(3) U.S. patent application Ser. No. 10/843,224, entitled TECHNIQUES FORANIMATING COMPLEX SCENES, filed concurrently with the presentapplication, now U.S. Pat. No. 7,064,761, issued Jun. 20, 2006.

BACKGROUND OF THE INVENTION

The present application relates to computer-generated imagery (CGI) andanimation, and more particularly to techniques for processing andmanipulating complex scenes.

The complexity of scenes in animated films keeps increasing with eachnew film as the number of objects in a scene and the level ofinteraction between the objects in a scene keeps increasing. Each objectin a scene is generally represented by a model. A model is generally acollection of geometric primitives, mathematical representations, andother information that is used to describe the shape and behavior of anobject. Accordingly, a scene may comprise multiple objects (eachrepresented by a model) interacting with each other.

Scenes are generally created by animators that specify the movement ofthe models and the interactions between the models. Conventionally, inorder for an animator to specify the animation for a particular scene,information related to all the models involved in the scene must beloaded into memory of the animator's computer or workstation. However,due to the larger number of models involved in scene of new films andthe richer interactions between the models, quite often the computingand memory resources of the animator's computer are inadequate to loadand process all the models involved in a scene. The increasingcomplexity of scenes has also impacted the manner in which scenes arerendered. Conventionally, when a 3D scene is rendered into a 2D image,all the models involved in the scene must be concurrently loaded intomemory of computer(s) allocated for the rendering process in order forthe scene to be rendered. However, many times for a complex scene, thememory resources of a renderer are not large enough to accommodate theinformation that must be loaded to render the scene. Nevertheless,animators and renderers must be able to animate and render such complexscenes with a minimum of overhead devoted to working around theabove-mentioned limitations.

One conventional technique that is used to try to resolve theabove-mentioned problems is to reduce the complexity of a scene. This isdone by representing each object in a scene at a low-fidelity thatconsumes less memory when loaded into computer memory of a computer. Thelow fidelity representations also evaluate more rapidly than the final(render) quality versions. However, even with the low-fidelity versionsof the models, there is a limit to the number of models that can beloaded into computer memory simultaneously. Further, as more and moremodels are loaded, the interactions between the models considerably slowdown thereby making the animation task slow and arduous. Also, extrawork on part of the animators is needed to build low-fidelity versionsof models and swap them in and out of scenes. This technique also doesnot help during the rendering process since final rendering generallyrequires the highest-fidelity (or full-fidelity) versions of the modelsto be used.

Another conventional technique that is commonly used to process largescenes is scene segmentation. In this technique, a user (e.g., ananimator) arbitrarily breaks up a scene into multiple sets of models,each of which becomes a scene of its own that is animated and renderedseparately. The resultant sub-scene images are then composited together.This decomposition is however artificial and an impediment to thecreative process. Further, it may be very difficult or even impossibleto break up a scene in which many models are interacting with oneanother.

In light of the above, techniques are desired that enable users such asanimators and other artists to process and manipulate large and complexscenes in an easy manner.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention provide techniques that facilitaterendering of a scene of arbitrary complexity without having to load allthe models involved in the scene simultaneously in computer memory. Arepresentation of the inputs and outputs of the various models involvedin the scene and the dependencies between the inputs and the outputs isprovided and used to facilitate the rendering process. A tessellator maytessellate the models without having to load information for all themodels concurrently into computer memory. The tessellated scene may thenbe rendered without having to load all the models in the sceneconcurrently into computer memory. Accordingly, a computer with aruntime memory that is insufficient to load all the models of the scenemay still be used to tessellate and render the scene.

According to an embodiment of the present invention, techniques areprovided for facilitating rendering of a scene comprising a plurality ofmodels. In an embodiment, an ordered list of model references isdetermined using a first representation, the first representationcomprising information identifying one or more inputs and outputs of themodels in the plurality of models and identifying dependencies betweenthe one or more inputs and outputs, each model reference identifying amodel from the plurality of models. The model references are thenprocessed in order of their position in the ordered list. The processingfor each model reference comprises: loading a model corresponding to themodel reference in computer memory, generating a result using the loadedmodel, and storing the generated result. The scene is rendered using thestored results, wherein the rendering is performed without loading allof the plurality of models concurrently in the computer memory.

The foregoing, together with other features, embodiments, and advantagesof the present invention, will become more apparent when referring tothe following specification, claims, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified block diagram of a computer system 100 that maybe used to practice an embodiment of the present invention;

FIG. 2 depicts a schematic representation of a Human model according toan embodiment of the present invention;

FIG. 3 is a simplified high-level flowchart depicting a method ofprocessing models in a scene to determine static intra-modeldependencies for the models according to an embodiment of the presentinvention;

FIG. 4 depicts a module that may be used to determine and store staticintra-model dependency information according to an embodiment of thepresent invention;

FIG. 5 is a simplified high-level flowchart depicting a method ofconstructing a proxy connectivity graph according to an embodiment ofthe present invention;

FIG. 6 depicts a schematic view of a proxy connection graph according toan embodiment of the present invention;

FIG. 7A is a simplified high-level flowchart depicting a method ofloading models for animation according to an embodiment of the presentinvention;

FIG. 7B is a simplified high-level flowchart 720 depicting a method ofunloading models from computer memory according to an embodiment of thepresent invention;

FIG. 8 depicts modules that may be used to construct and manipulate aproxy connectivity graph according to an embodiment of the presentinvention;

FIG. 9 is a simplified high-level flowchart depicting processingperformed to facilitate rendering of a scene according to an embodimentof the present invention;

FIG. 10 depicts modules that may be used to perform the processingdepicted in FIG. 9 according to an embodiment of the present invention;and

FIGS. 11A through 11F depict various stages of clustering nodes in alinear ordered list according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, for the purposes of explanation, specificdetails are set forth in order to provide a thorough understanding ofthe invention. However, it will be apparent that the invention may bepracticed without these specific details.

Embodiments of the present invention provide techniques for processinglarge and complex scenes. Artists such as animators, renderers, andlayout artists can work with complex scenes in a natural way withouthaving to worry about the complexities of the scene, available memoryand/or computing resources, scene segmentation, different fidelityrepresentations, and dependencies between models in a scene. FIG. 1 is asimplified block diagram of a computer system 100 that may be used topractice an embodiment of the present invention. For example, systemssuch as computer system 100 may be used by an animator to specify orassemble a scene. Systems such as computer system 100 may also be usedto render scenes.

As shown in FIG. 1, computer system 100 includes a processor 102 thatcommunicates with a number of peripheral devices via a bus subsystem104. These peripheral devices may include a storage subsystem 106,comprising a memory subsystem 108 and a file storage subsystem 110, userinterface input devices 112, user interface output devices 114, and anetwork interface subsystem 116. The input and output devices allow auser, such as the administrator, to interact with computer system 100.

Network interface subsystem 116 provides an interface to other computersystems, and networks. Network interface subsystem 116 serves as aninterface for receiving data from other sources and for transmittingdata to other sources from computer system 100. Embodiments of networkinterface subsystem 116 include an Ethernet card, a modem (telephone,satellite, cable, ISDN, etc.), (asynchronous) digital subscriber line(DSL) units, and the like.

User interface input devices 112 may include a keyboard, pointingdevices such as a mouse, trackball, touchpad, or graphics tablet, ascanner, a barcode scanner, a touchscreen incorporated into the display,audio input devices such as voice recognition systems, microphones, andother types of input devices. In general, use of the term “input device”is intended to include all possible types of devices and mechanisms forinputting information to computer system 100.

User interface output devices 114 may include a display subsystem, aprinter, a fax machine, or non-visual displays such as audio outputdevices, etc. The display subsystem may be a cathode ray tube (CRT), aflat-panel device such as a liquid crystal display (LCD), or aprojection device. In general, use of the term “output device” isintended to include all possible types of devices and mechanisms foroutputting information from computer system 100.

Storage subsystem 106 may be configured to store the basic programmingand data constructs that provide the functionality of the presentinvention. For example, according to an embodiment of the presentinvention, software code modules (or instructions) implementing thefunctionality of the present invention may be stored in storagesubsystem 106. These software modules may be executed by processor(s)102. Storage subsystem 106 may also provide a repository for storingdata used in accordance with the present invention. Storage subsystem106 may comprise memory subsystem 108 and file/disk storage subsystem110.

Memory subsystem 108 may include a number of memories including a mainrandom access memory (RAM) 118 for storage of instructions and dataduring program execution and a read only memory (ROM) 120 in which fixedinstructions are stored. RAM is generally semiconductor-based memorythat can be read and written by processor 102. The storage locations canbe accessed in any order. The term RAM is generally understood to referto volatile memory that can be written to as well as read. There arevarious different types of RAM. For purposes of this application,references to information being loaded or unloaded from compute memoryrefer to loading or unloading the information from RAM (or any othervolatile computer memory used by a program or process during execution)of a computer.

File storage subsystem 110 provides persistent (non-volatile) storageand caching for program and data files, and may include a hard diskdrive, a floppy disk drive along with associated removable media, aCompact Disk Read Only Memory (CD-ROM) drive, an optical drive,removable media cartridges, and other like storage media.

Bus subsystem 104 provides a mechanism for letting the variouscomponents and subsystems of computer system 100 communicate with eachother as intended. Although bus subsystem 104 is shown schematically asa single bus, alternative embodiments of the bus subsystem may utilizemultiple busses.

Computer system 100 can be of various types including a personalcomputer, a portable computer, a workstation, a network computer, amainframe, a kiosk, or any other data processing system. Due to theever-changing nature of computers and networks, the description ofcomputer system 100 depicted in FIG. 1 is intended only as a specificexample for purposes of illustrating the preferred embodiment of thecomputer system. Many other configurations having more or fewercomponents than the system depicted in FIG. 1 are possible.

A scene refers to a set of frames shot (generally continuously) using acamera between a start time and an end time. Scenes are generallyspecified (or assembled) by one or more animators. Information storedfor a scene may include camera angles information, lighting information,the starting time and ending time for the scene, information identifyingthe models in a scene, descriptions of the models, description of howthe models interact or move within a scene, and other information. Inone embodiment, the information may be stored in a scene data file, as ascene graph, etc. The stored scene information is then used to renderthe scene so that is can be viewed by animators, and others. The processof specifying a scene is typically an iterative process where ananimator assembles a scene, views the rendered scene, makes changes tothe scene if necessary, and repeats the process.

A scene may include a number of objects or components possiblyinteracting with one another. Each object may be represented by a model.A model is generally a collection of geometric primitives, mathematicalrepresentations, and other information that is used to describe theshape and behavior of an object. Information specifying a model may bereferred to as model information.

Each model may comprise (or is characterized by) a set of inputparameters (or inputs) and a set of outputs. Values may be provided tothe inputs to position and configure the geometric primitives of themodel. Values of the outputs may represent the orientation and positionof the geometric primitives of the model. An input of a model mayreceive its value from (and thus depend upon) one or more outputs ofother models or of the same model. An output of a model may receive itsvalue from (and thus depend on) one or more inputs of the model, i.e.,values of the inputs are used to compute the output's value.Dependencies may also be classified as intra-model or inter-model. Whenan output of a model depends on one or more inputs of the same model, orwhen an input of a model depends on one or more outputs of the samemodel, such a dependency is referred to as an intra-model dependency (orintra-model dataflow connection). When an input of one model depends onan output of another model, the dependency is referred to as aninter-model dependency (or inter-model dataflow connection). Intra andinter-model dependencies may arise from constraints placed by theanimator on a model and between models.

Generally, a model A depends on another model B if model A makes use ofdata provided by model B. For example, model A depends directly on modelB if an input or output of model A uses a value of an input or output ofmodel B. Model B may in turn depend on other models, which may in turndepend on other models, and so on. Model A indirectly depends on modelsthat model B depends on, models that those models depend on, and so on.For example, if model A depends on model B which depends on model Cwhich in turn depends on model D, then model A depends directly on modelB and indirectly on models C and D. The set of models that model Adepends on includes all models on which model A directly or indirectlydepends on. For the above example, the set of models on which model Adepends includes models B, C, and D. The set may also be referred to asthe transitive closure set of models that model A depends on.

In the above example, where model A depends on model B which depends onmodel C which in turn depends on model D, models A, B, and C representthe models that depend upon model D. Accordingly, a set comprisingmodels A, B, and C represents a transitive closure set of models thatdepend on model D.

Two models may exhibit a dependency where model A depends on model B andmodel B depends on model A, although there can be no input/output pairanywhere in a scene at any particular time in which the input depends onthe output and the output also depends on the input, as that would causethe output's value to be incomputable

Dependencies may either be static or dynamic in nature. A staticdependency is a dependency that is fixed or time invariant over a periodof time. A dynamic dependency is a dependency that may be time variantover a period of time. An intra-model dependency where an output of amodel depends on one or more inputs of the same model is generally astatic dependency that is configured when the model is built.

Consider a scene in which a human with a cell-phone clipped to his beltfollows the progress of the moon with his gaze, while leaning with hisright hand on a rail and holding a cup of coffee in his left hand. Thescene may include several models including models representing thehuman, phone, rail, cup, moon, etc. Each model may have a set of inputsand outputs. FIG. 2 depicts a schematic representation of a Human model200 according to an embodiment of the present invention. Human model 200comprises three inputs (shown by diamonds and having an “In” labelending) and three outputs (shown by circles and having an “Out” labelending). The inputs include waistIn (HWI), headOrientIn (HHI), andrHandIn (HRI). The outputs include waistOut (HWO), neckOut (HNO), andIHandOut (HLO).

In FIG. 2, dotted arrows 202 are used to represent the staticintra-model dependencies where an output of the model depends on one ormore inputs of the same model. The arrows point in the direction of thedependency. For example, as shown, neckOut (HNO) depends on waistIn(HWI), IHandOut (HLO) depends on waistIn (HWI) and waistOut (HWO)depends on waistIn (HWI). The inter-model dependencies for the scene aredepicted in FIG. 6 and described below.

According to an embodiment of the present invention, in order tofacilitate processing of large scenes, the model information for eachmodel included in a scene is analyzed to determine the staticintra-model dependencies (or static intra-model dataflow connections)between inputs and outputs of the model. The static intra-modeldependency information (or static intra-model dataflow connectioninformation) for a model is persistently cached. Various differenttechniques may be used for caching the static intra-model dependencyinformation. According to an embodiment of the present invention, thestatic intra-model dependency information for a model is stored in afile (referred to as the “hook file” or “hook”) associated with themodel. A hook file for a model may also store other information for themodel besides dependency information. The information may be stored invarious formats. In one embodiment, the information is stored in theform of a directed graph such that the nodes of the graph represent (areproxies for) the inputs or outputs of a model and the links between thenodes represent the static intra-model dataflow connections. Other datastructures may also be used in alternative embodiments.

An example of static intra-model dependency information that may bestored in a hook file is shown below. The hook file comprising theinformation below is associated with a Human model (different from themodel depicted in FIG. 2).

hook “Human” { inputHandles = waistIn, headOrientIn, rHandIn, lHandIn;outputHandles = waistOut waistIn, neckOut waistIn, rHandOut rHandInwaistIn, lHandOut lHandIn waistIn; }As shown above, the “inputHandles” identifies the inputs of the Humanmodel, namely, waistIn, headOrientIn, rHandIn, and rHandIn. The“outputHandles” identifies the outputs of the model and also theirstatic intra-model dependencies. As shown above, there are four outputs,namely, waistOut, neckOut, rHandOut, and IHandout. The output waistOutdepends on input waistIn, the output neckOut depends on input waistIn,the output rHandOut depends on inputs rHandIn and waistIn, and theoutput IHandOut depends on inputs IHandIn and waistIn.

FIG. 3 is a simplified high-level flowchart 300 depicting a method ofprocessing models in a scene to determine static intra-modeldependencies for the models according to an embodiment of the presentinvention. The method depicted in FIG. 3 may be performed by softwaremodules executed by a processor, hardware modules, or combinationsthereof. Flowchart 300 depicted in FIG. 3 is merely illustrative of anembodiment of the present invention and is not intended to limit thescope of the present invention. Other variations, modifications, andalternatives are also within the scope of the present invention. Themethod depicted in FIG. 3 may be adapted to work with differentimplementation constraints.

As depicted in FIG. 3, processing is initiated upon receivinginformation identifying a scene (step 302). The scene information isthen analyzed to determine the one or more models involved in the scene(step 304). As is known in the art, the scene information may be storedin various forms such as a scene graph, etc. and may be used to performthe analysis in step 302. Model information for each model identified instep 304 is then accessed and the static intra-model dependencies of theoutputs of a model on the inputs of the model are determined for eachmodel (step 306). According to an embodiment of the present invention,as part of step 306, each model is loaded into computer memory (i.e.,the model information for the model is loaded into computer memory), thedependencies determined using the loaded model information, and then themodel may be unloaded from the computer memory. Accordingly, all themodels (i.e., the model information for all the models) do not need tobe loaded into computer memory concurrently to determine staticintra-model dependency information. For each model, the staticintra-model dependency information determined for the model in step 306is then persistently stored, for example in a hook file that isassociated with the model (step 308).

In alternative embodiments, instead of doing the analysis on a scenebasis as depicted in FIG. 3, the intra-model static dependencies for amodel may be determined and stored in a hook file associated with themodel when the model is built or configured. Accordingly, whenever themodel is subsequently used, the static intra-model dependencyinformation for the model is available for processing from the hook fileassociated with the model. If a model is modified (i.e., modelinformation for the model is changed), the dependency information storedin the hook file associated with the model may be updated to reflect thedependency changes, if any, resulting from the changed modelinformation.

In other embodiments, a user may select or specify the models for whichintra-model dependency information is to be computed. In theseembodiments, for each model specified by the user, the model informationfor the model is analyzed to determine the static intra-modeldependencies for the model, and the dependency information is cached ina hook file associated with the model.

FIG. 4 depicts a module that may be used to determine and store staticintra-model dependency information according to an embodiment of thepresent invention. As depicted in FIG. 4, an intra-model dependencyanalyzer 400 may be provided. Intra-model dependency analyzer 400 takesas input model information 402 for a model, analyzes model information402 to determine the static intra-model dependency information, andstores the dependency information in a hook file 404 associated with themodel or with the model information. Intra-model dependency analyzer 400may be implemented using software, hardware, or combinations thereof.

As an artist works on a scene, the authored data that represents theanimations for the model in the scene is stored in a cue. The cue storesinformation describing the inter- and intra-model dataflow connectionsamong the models for the scene. The cue stores time varyingspecification for inputs of models that is used to pose the models. Cuesmay store time-varying (i.e., dynamic) dependencies between the models,including the intra- and inter-model dependencies between the inputs andoutputs of models. According to an embodiment of the present invention,based upon the cue information for a scene and the given the hooks forthe models in the scene, a representation is constructed in memory forthe scene. The memory representation that is constructed storesinformation specifying the inter- and intra-model dependencies (i.e.,the inter- and intra-model dataflow connections) between the variousinputs and outputs of the models included in a scene. The memoryrepresentation encapsulates the relationships between the models in ascene.

In one embodiment, the memory representation is a directed graph,referred to as a proxy connectivity graph. Other data structures mayalso be used for the memory representation in alternative embodiments.The nodes of the proxy connectivity graph represent (are proxies for)inputs and outputs of the models of the scene and the directed linksbetween the nodes represent the dataflow connections or dependencies(both static and dynamic inter-model and intra-model dependencies)between the inputs and outputs of the models. When referring to a proxyconnectivity graph, the words proxy and node will be usedinterchangeably. The proxy connectivity graph is built based upon thehooks and cues that are (in comparison to models) inexpensive to loadand keep resident in memory for purposes of constructing the proxyconnectivity graph. The proxy connectivity graph itself is lightweightcompared to loading all the models and thus can be easily loaded andkept resident in computer memory for further processing.

The proxy connectivity graph also stores information about the mappingof dataflow connections to corresponding authored elements in the cue. Auser (e.g., an animator) may use the proxy graph to create and deleteconnections between models without requiring all the models to beloaded, using the same user interface components, with deep analysiscapabilities available. For instance, the user can verify that arequested dataflow connection is legal (e.g., creates no cycles in theconnectivity graph) from the proxy graph. The user can performoperations such as establishing and breaking time-varying relationshipsbetween models by making changes to the proxy connectivity graph withoutneeding to load all models simultaneously. The user can automaticallypropagate changes to the proxy connectivity graph back into the actualscene graph or scene information itself.

FIG. 5 is a simplified high-level flowchart 500 depicting a method ofconstructing a proxy connectivity graph according to an embodiment ofthe present invention. The method depicted in FIG. 5 may be performed bysoftware modules executed by a processor, hardware modules, orcombinations thereof. Flowchart 500 depicted in FIG. 5 is merelyillustrative of an embodiment of the present invention and is notintended to limit the scope of the present invention. Other variations,modifications, and alternatives are also within the scope of the presentinvention. The method depicted in FIG. 5 may be adapted to work withdifferent implementation constraints.

As depicted in FIG. 5, processing is initiated upon receivinginformation identifying a time interval (step 502). The time intervalmay be characterized by a start time (Ts) and an end time (Te).According to an embodiment of the present invention, the start time (Ts)corresponds to the start time of a scene and the end time (Te)corresponds to the end time of the scene. In alternative embodiments,the time interval may also correspond to a subset of the scene.

The hooks information (i.e., information stored in the hook filesassociated with the model) for the models included in the scene and thecues information for the scene is then accessed and loaded into memory(step 504). As previously described, the hook information for a modelstores the static intra-model dependencies for the model. The cuesinformation comprises information describing the time-varying inter andintra-model dataflow connections among the models for the scene. The cueinformation stores time varying specification for inputs of models thatis used to pose the models; these specifications determine thetime-varying (i.e., dynamic) dependencies between inputs and outputs ofmodels. Any arbitrary type of dataflow connection or dependency may bedefined including two-way dataflow connections between models, transferof data from a model to itself, transfer of data from a model to anothermodel, and the like. Both the hooks and the cues are lightweight andinexpensive to load and keep resident in memory of a computer, ascompared to loading the models themselves (i.e., loading the modelinformation for all the models).

A proxy connectivity graph is constructed based upon the hooksinformation accessed and the cues information accessed and loaded instep 504 (step 506). The nodes of the proxy connectivity graph representthe inputs and outputs of the models included in the scene and the linksbetween the nodes represent dataflow connections or dependencies betweenthe inputs and the outputs for the time interval. The proxy connectivitygraph is built and kept resident in computer memory for further use.

An example of a proxy connectivity graph is depicted in FIG. 6. FIG. 6depicts a schematic view of a proxy connection graph according to anembodiment of the present invention for a scene in which a human with acell-phone clipped to his belt follows the progress of the moon with hisgaze, while leaning with his right hand on a rail and holding a cup ofcoffee in his left hand. The scene includes several models including aHuman 602, a Phone 604, a Rail 606, a Cup 608, a Targeter 610, and aMoon 612. Each model has one or more inputs and/or outputs. For purposesof clarity, inputs have labels ending in “In” and are depicted usingdiamonds while outputs have labels ending in “Out” and are depictedusing circles. The input and output nodes are also labeled using threeletter labels (e.g., HWO, PCI). The first letter of the label identifiesthe model to which the input/output belongs, for example, “H” for Human,“P” for Phone, “M” for Moon, “C” for Cup, “T” for Targeter, and “R” forRail.

As shown in FIG. 6, the scene results in a number of intra andinter-model dependencies. The arrows between the inputs and outputsdepict the dependencies for the scene (which is opposite to thedirection of dataflow). The arrows point in the direction of thedependency. Accordingly, in FIG. 6, a first node depends on a secondnode if an arrow originates (i.e., tail of the arrow) from the firstnode and points to the second node (i.e., head of the arrow points tothe second node). In FIG. 6, static intra-model dependencies aredepicted using dotted arrow lines while inter-model dependencies aredepicted using solid arrow lines. For example, the coffee cup isconstrained to the human's left hand because anywhere the human movesthe hand, the cup must follow, and therefore the cup effectively takesits position from the hand. Accordingly, as depicted in FIG. 6, thehandlein input of the Cup depends upon the IHandOut output of the Humanmodel. The nodes representing the inputs and outputs along with thearrows represent the proxy connectivity graph for the scene.

Targeter model 610 does not represent a physical object that will appearin the rendered scene, but rather a separate unit of computation notbuilt into any model and therefore usable by any model. Targeter model610 encapsulates the computation that will orient an object to point ata second target object, given the first object's own pivot (center ofrotation). This is a common computation in computer graphics, and isfrequently embedded directly in models that require the computation. Itis a simple example of a pattern that may be commonly used for other,much more complicated computations.

A proxy connectivity graph for a scene may be used for several differentapplications. For example, the proxy graph may be used to enable a user(e.g., an animator) to specify animation for a large complex scene suchthat only those models that are needed for the animation are loaded incomputer memory—all the models involved in the scene do not have to beloaded. According to an embodiment of the present invention, the proxyconnectivity graph for a scene is used to determine the models that arerequired, either directly or indirectly, for the animation. For example,for any given model in a scene, the proxy connectivity graph for thescene can be walked or traversed to determine which other models wouldneed to be loaded in order for the given model to be able to evaluateitself correctly. The proxy graph is traversed to determine a transitiveclosure set (or recursive closure set) comprising one or more modelsfrom the scene, including the given model, that are needed for the givenmodel to evaluate correctly. This allows users to easily load all andonly the models necessary for them to work, even in a very large scenewith many models without having to load all the models concurrently incomputer memory.

For example, if an animator wishes to animate a particular model (or setof models) for a large scene, only a minimal subset of the scene'smodels that are needed to properly animate the particular modelspecified by the animator need to be loaded into the memory of thesystem used by the animator. The animator need only identify the modelsfrom a scene in which the animator is interested, and an embodiment ofthe present invention automatically constructs the proxy connectivitygraph (if it is not already constructed and resident in memory) for thescene based on the cues for the scene and based upon hooks of the modelsincluded in the scene, traverses the proxy graph to determine a minimalset (transitive or recursive closure set) of models that are needed tobe concurrently loaded in memory in order for the user-identified modelsto evaluate correctly, and loads the determined minimal set of modelsand the user-identified models into the memory of the computer used bythe user. All the models do not have to be loaded. Accordingly, theproxy connectivity graph for a scene provides the ability toautomatically and dynamically manage the number of models that need tobe loaded into a computer memory for processing of the scene.

FIG. 7A is a simplified high-level flowchart 700 depicting a method ofloading models for animation according to an embodiment of the presentinvention. The method depicted in FIG. 7A may be performed by softwaremodules executed by a processor, hardware modules, or combinationsthereof. Flowchart 700 depicted in FIG. 7A is merely illustrative of anembodiment of the present invention and is not intended to limit thescope of the present invention. Other variations, modifications, andalternatives are also within the scope of the present invention. Themethod depicted in FIG. 7A may be adapted to work with differentimplementation constraints.

As depicted in FIG. 7A, information may be received from an animatoridentifying a particular model (or multiple models) that the animator isinterested in animating for a scene (step 702). Based upon theinformation received in step 702, the proxy connectivity graph for thescene that is resident in computer memory is traversed to determine aminimal set of models including models that are needed to be loaded inorder for the particular model specified by the animator to evaluatecorrectly (step 704). The minimal set of models represents thetransitive closure (or recursive closure) set of models with respect todependencies of the particular model specified by the animator that areneeded for the user-specified model to evaluate correctly. For example,the minimal set of models may include a first set of models on which theparticular animator-specified model depends, a second set of models onwhich the first set of models depends on, and so on recursively.

According to an embodiment of the present invention, as part of step704, each input of the model to be loaded is considered in turn. Foreach input being considered, a node in the proxy graph representing theinput is determined. Using the input node as the starting point andusing the directed links of the proxy graph, the proxy graph istraversed or walked in the direction of the dependencies to identifynodes on which the input node being examined depends upon, eitherdirectly or indirectly (e.g., if a directed arrow in the proxy graphfrom a first node to a second node indicates that the first node dependson or uses a value of the second node, then the graph is walked in thedirection of the directed arrows). As a result of the traversal, atransitive closure set of all inputs and outputs on which the input ofthe particular model being considered depends upon is determined. Arecord is kept of which models' nodes are encountered during the walk.The union of the models recorded for each “walk” for each input of theparticular node represents the minimal set of models to be loaded incomputer memory simultaneously for the particular model to evaluatecorrectly. Any model whose input or output is encountered during thetraversals is included in the minimal set of models.

The set of models determined in step 704 and the particular modelspecified by the user are then loaded in the computer memory of theanimator's computer to enable accurate animation of the particular modelspecified by the animator in step 702 (step 706). The animator may thenanimate the particular model. In this manner, an animator can performanimation for a large scene comprising several models without having toload all the models for the scene concurrently into the computer memory.Embodiments of the present invention thus enable a user to animatemodels from a large complex scene even if the animator's computer'smemory is not large enough to load all the models for the scene. Byloading only the models that are required to properly animate aparticular model, the proper animation of the particular model isassured without having to load all the models in the scene.

For example, let's assume that the animator wishes to animate the Humanmodel in the graph depicted in FIG. 6 using animation software. Theanimator selects the Human for loading, and a system configuredaccording to the teachings of the present invention determines whatother models must also be loaded in order for the Human to movecorrectly while the animator animates it. The proxy connectivity graphis traversed to determine the dependencies. In one embodiment, each noderepresenting an input of the Human model is examined in turn, and theproxy graph is walked from each node, recording which other models'nodesare encountered. For the Human model, the models that will beencountered by walking the proxy graph depicted in FIG. 6 are Targeter,Rail, and Moon. The system therefore loads the Human, Rail, Targeter,and Moon models. The Phone and Cup models are not loaded as the Humanmodel does not depend on them, but rather the opposite. If the animatorhad selected the Cup model for animation instead, then Cup, Human, Rail,Target, and Moon models would be loaded.

From the set of models that have been loaded into a computer memory, theuser may also specify a particular model to be unloaded from thecomputer memory. Since other models may depend on the model that is tobe unloaded, the other models should also be unloaded to preventincorrect animation of those models. According to an embodiment of thepresent invention, the proxy connectivity graph may be used to determinewhich other models must be unloaded.

FIG. 7B is a simplified high-level flowchart 720 depicting a method ofunloading models from computer memory according to an embodiment of thepresent invention. The method depicted in FIG. 7B may be performed bysoftware modules executed by a processor, hardware modules, orcombinations thereof. Flowchart 720 depicted in FIG. 7B is merelyillustrative of an embodiment of the present invention and is notintended to limit the scope of the present invention. Other variations,modifications, and alternatives are also within the scope of the presentinvention. The method depicted in FIG. 7B may be adapted to work withdifferent implementation constraints.

As depicted in FIG. 7B, information may be received from a user (e.g.,an animator) identifying a model to be unloaded from the memory of acomputer (step 722). In response to the user request, the proxyconnectivity graph is traversed to identify a transitive closure set ofall models that depend, either directly or indirectly, on the model tobe unloaded (step 724). According to an embodiment of the presentinvention, as part of step 724, a node from the proxy graph isdetermined for each output of the model to be unloaded. Each noderepresenting an output of the particular model to be unloaded isexamined in turn. Using the output node (i.e., a node representing anoutput) being examined as the starting point and using the directedlinks of the proxy graph, the proxy graph is traversed or walked in theopposite direction of the dependencies (i.e., if a directed arrow in theproxy graph from a first node to a second node indicates that the firstnode depends on or uses the value of the second node, then the graph iswalked in the direction from a node pointed to by an arrow to the nodeconnected to the source of the arrow) to identify nodes that depend,either directly or indirectly, on the node representing the output ofthe model being examined. Accordingly, the directed links between thenodes are used to walk the graph. As a result of the traversal, atransitive closure set of all inputs and outputs that depend on theoutput of the model to be unloaded is determined. A record is kept ofwhich models' nodes are encountered during the “reverse” walk (i.e.,models whose inputs or outputs are included in the transitive closureset). The union of the models recorded for each reverse walk for eachoutput of the model to be unloaded represents the minimal set of modelsto be unloaded from the computer memory to prevent faulty animation. Anymodel whose input or output is encountered during the traversals isincluded in the minimal set of models.

From the models determined in 724, a set of models that are presentlyloaded in the computer memory are determined (step 726). The modelsdetermined in 726 and the user-specified model to be unloaded are thenunloaded from the computer memory (step 728). In this manner,embodiments of the present invention prevent “incorrect” animation for aparticular model that may result if a model on which the particularmodel depends is not loaded and available for processing.

FIG. 8 depicts modules that may be used to construct and manipulate aproxy connectivity graph according to an embodiment of the presentinvention. The modules depicted in FIG. 8 may be implemented assoftware, in hardware, or combinations thereof. Other variations,modifications, and alternatives are also within the scope of the presentinvention.

As depicted in FIG. 8, a proxy graph constructor and analyzer (PGCA)module 802 is provided that takes as input hooks 806 associated withmodels 804 for a scene and cues information 808 for a scene and builds aproxy connectivity graph 810. A model loader 812 may receive information814 from a user (e.g., an animator) specifying one or more models thatare of interest to the user. Model loader 812 is configured to “walk” ortraverse proxy connectivity graph 810 to determine a minimal set ofmodels 816 (transitive closure set) that is needed for the user-selectedmodels to evaluate properly. Model loader 812 then loads the models inthe minimal set 816 in memory 818 of computer 820 used by the user.Model loader 812 may also be configured to unload models from memoryaccording to the flowchart depicted in FIG. 7B.

The user may also make changes 822 to the scene using constraints editor824. For example, a user may make or break constraints between modelsusing constraints editor 824, thereby changing the intra and/or intermodel dependencies. The changes are stored in cues information 808 forthe scene. The changes are also provided to PGCA 802 which is configuredto modify proxy connectivity graph 810 to reflect the changes. In thismanner, a user can make changes to the scene without having to load allthe models for the scene concurrently in computer memory.

The proxy connectivity graph built according to the teachings of thepresent invention also facilitates rendering of large scenes. Manycommercial renderers, including RenderMan® from Pixar™ are generallycapable of rendering large scenes if one is able to provide them a fullytessellated (mathematical description of models composed of adjoining 3Dpolygons) description of the scene in a single and consistent coordinatesystem. However, due to the large and unwieldy nature of tessellatedrepresentations, it is generally impractical to store scenes in atessellated description as they are being animated. Consequently,tessellation of a scene has to generally be performed before the scenecan be rendered.

To make tessellation tractable for a scene of arbitrary size andcomplexity, techniques are needed for automatically decomposing thescene into units or sets that can be tessellated individually. However,due to dependencies between models in a scene that result from animatedinteractions in the scene, it may not be possible to cleave the scenegraph into such disjoint sets. According to an embodiment of the presentinvention, the proxy connectivity graph is used to facilitate therendering process. An optimizing graph linearizer uses the proxy graphto trade off model loads for smaller sets (or clusters) of models thatmust be simultaneously loaded. Since loading models is expensive (addssignificantly to the rendering time), the linearizer performsoptimization to minimize the number of times each model must be loadedin order to be tessellated without concurrently loading all of themodels upon which it depends.

FIG. 9 is a simplified high-level flowchart 900 depicting processingperformed to facilitate rendering of a scene according to an embodimentof the present invention. The method depicted in FIG. 9 may be performedby software modules executed by a processor, hardware modules, orcombinations thereof. Flowchart 900 depicted in FIG. 9 is merelyillustrative of an embodiment of the present invention and is notintended to limit the scope of the present invention. Other variations,modifications, and alternatives are also within the scope of the presentinvention. The method depicted in FIG. 9 may be adapted to work withdifferent implementation constraints.

As depicted in FIG. 9, the linearizer “unrolls” the proxy connectivitygraph into an ordered linear list of nodes presenting inputs and outputs(step 902). For the description below, a node is a proxy for an input oroutput of a model; accordingly, an ordered list of inputs and outputsrepresented by the proxy graph is determined in 902 and used forsubsequent processing. The unrolling operation is performed such thatthe following condition (“the invariant condition”) is always satisfied:in the ordered list of nodes (inputs and outputs), all nodes that aparticular node depends upon appear in the ordered list before theparticular node. For example, if a particular node representing an input(also referred to as an input node) of a model depends on two nodesrepresenting outputs (or output nodes) of another model (implying thatthe input depends on the two outputs), then in order to satisfy theinvariant condition, the two output nodes are positioned before theparticular input node in the ordered list of nodes. If the ordered listof nodes is considered to be ordered from left to right, then theinvariant condition is satisfied if all nodes that a particular nodedepends upon appear in the ordered list to the left of the particularnode.

The invariant condition guarantees that when the nodes are evaluated (asdescribed below in step 910), all the data that is needed to evaluate aparticular node is cached and available before the particular node isevaluated. This ensures proper evaluation of all the nodes in the proxyconnectivity graph and the proper rendering of the scene without havingto load all the models into computer memory simultaneously.

Clusters (or sets) of nodes belonging to the same model are thendetermined in the ordered list of nodes (step 903). A cluster comprisesone or more nodes of the same model that are contiguous in the orderedlist of nodes.

The linearizer then iteratively reorders the nodes in the ordered listto minimize the number of “clusters” (or sets) of nodes that belong tothe same model, while ensuring that the invariant condition is satisfiedat every stage of the clustering (step 904). Clustering is performed instep 904 to optimize the evaluation process by reducing the number oftimes that a model must be loaded in order to evaluate the nodes in theordered list of nodes corresponding to the model. If clustering were notperformed, then if “n” nodes of a model were scattered into “m”noncontiguous clusters, the model would need to be loaded “m” times inorder to evaluate the nodes of the model. Since loading and unloading amodel is expensive in terms of use of computing and memory resources,the goal of the processing performed in step 904 is to cluster nodes ofa model into a minimal number of clusters (optimally into one cluster ifpossible, which implies that the model corresponding to the nodes in thecluster need be loaded only once to evaluate all of the nodes of themodel).

The processing in step 904 is repeated until no further reduction in thenumber of clusters is possible by reordering the nodes without violatingthe invariant condition. Accordingly, a check is made to determine ifany further reordering can be performed (step 906). If no furtherreordering can be performed, it indicates that an optimal ordering ofthe nodes has been reached that minimizes the number of times each modelmust be loaded for the evaluation processing.

An ordered list of model references is then determined based upon theordering of the clusters in the ordered list of nodes generated at theend of the reordering (step 908).

The models are then evaluated or posed based upon information in the cue(step 910). Posing includes providing a set of values (which may bestored in the cue) to the inputs of a model that position and configurethe geometric primitives of the model. The time-varying specification ofposes results in animation for a scene. As part of step 910, each modelis loaded into a computer's memory according to the order specified bythe ordered list of model references determined in step 908. The loadedmodel is then used to evaluate the nodes (inputs and/or outputs) in thecluster corresponding to the model. The results of each evaluation arecached or stored. Caching the results for each cluster guarantees thatinformation needed for evaluating a particular node (input or output) ofa model is available prior to the evaluation of the node. Previouslycached results may also be used for evaluating or posing of a model in910.

The tessellator then produces tessellations for each model based uponthe data cached in step 910 (step 912). The tessellator may tessellateeach model individually. The caching of information performed in step910 enables the models to be tessellated in any arbitrary order.According to an embodiment of the present invention, the tessellatorsequentially loads each model individually and uses the informationcached in 910 for the model and its dependencies to producetessellations for the loaded model. The tessellated data may then berendered by a renderer (step 914).

In this manner, a scene of arbitrary complexity can be rendered withoutneeding to load all the models in the scene simultaneously in the memoryof a computer. The tessellator can also tessellate the models on aper-model basis without needing to load information for all themodels—the cached data is used for the tessellation instead.Accordingly, a computer with a memory that is insufficient to load allthe models of the scene may still be used to tessellate and render thescene.

If tessellating an interval of time in the scene (which is the commoncase), the linearizer in general may be applied for multiplesub-intervals—whenever the dataflow relationships change sufficientlywithin the interval such that a single unrolling of the proxy graph isno longer sufficient to evaluate the entire interval. By performing abisection search on the interval and considering all dataflowconnections within the interval, we can determine the smallest set ofintervals for which the linearizer must be individually applied.

FIG. 10 depicts modules that may be used to perform the processingdepicted in FIG. 9 according to an embodiment of the present invention.The modules depicted in FIG. 10 may be implemented as software, inhardware, or combinations thereof. Other variations, modifications, andalternatives are also within the scope of the present invention.

As depicted in FIG. 10, a proxy graph constructor and analyzer (PGCA)module 1002 is provided that takes as input hooks 1006 associated withmodels 1004 for a scene and cues 1008 for the scene and builds a proxyconnectivity graph 1010. An optimizing graph linearizer 1012 traversesand unrolls proxy connectivity graph 1010 to generate a linearizedordered list of nodes. Graph linearizer 1012 is configured toiteratively reorder the nodes in the ordered list with the goal ofminimizing the number of clusters of nodes belonging to a model whilesatisfying the invariant condition. After no more re-orderings can beperformed, linearizer 1012 may determine a list of ordered modelreferences based upon the ordered list of clusters. The ordered list ofmodel references along with intervals 1014 may then be forwarded toposer 1016.

Poser 1016 is configured to evaluate or pose the models based uponinformation received from linearizer 1012, cues imformation 1008, andmodel information 1004. As part of the posing, each model is loaded intoa computer's memory according to the order specified by the ordered listof model references. The loaded model is then used to evaluate the nodesin a cluster corresponding to the model. The results of each evaluationare cached. The models are loaded and unloaded one by one per theordering in the model list of references until all the models have beenevaluated.

The cached data is then used by tessellator 1018 to produce tessellatedrenderable data 1020. Tessellator 1016 is configured to tessellate eachmodel individually using the cached information. The models can betessellated in any order. Renderable data 1020 is then communicated to arenderer 1022 (e.g., RenderMan® from Pixar) for rendering.

In this manner, scenes of arbitrary complexity can be rendered with nointervention or decomposition required from the user. Embodiments of thepresent invention are able to linearize the scene graph in such a waythat no more than a few models need ever be resident in the memory ofthe rendering system simultaneously. The tessellator can tessellate amodel without needing to load all the other models concurrently inmemory as the information needed for tessellating the model is cachedand available to the tessellator.

EXAMPLE

This section uses an example to describe the process of unrolling aproxy graph to produce a linear ordered list of nodes and reordering ofnodes in the ordered list to minimize the number of clusters of nodesrepresenting inputs or outputs of the same model (referred to as nodesbelonging to the same model) according to an embodiment of the presentinvention. The example is not intended to limit the scope of the presentinvention as recited in the claims. The proxy connectivity graphdepicted in FIG. 6 is used to illustrate the example.

(1) Obtain a Valid Linear Ordered List (Valid Linearization)

The proxy connectivity graph (depicted in FIG. 6) is unrolled togenerate a linear ordered list of nodes. The unrolling is performed suchthat the invariant condition (that all nodes that a particular nodedepends upon appear in the ordered list before the particular node) issatisfied. In one embodiment, all proxy nodes of the proxy connectivitygraph are iterated through placing the nodes at the end of the(initially empty) linearized list after all nodes upon which they dependhave already been added to the list. With proper marking of nodes, thiscan be performed efficiently in linear time with a simple recursivefunction.

One typical unrolling of the proxy graph depicted in FIG. 6 is depictedin FIG. 11A, in which arrows indicate dependency. The linear orderedlist depicted in FIG. 11A comprises thirteen nodes corresponding toinput and output nodes in the proxy connectivity graph depicted in FIG.6. The nodes are ordered left to right. The arrows indicate thedependencies of the nodes. The nodes in the ordered list depicted inFIG. 11A are clustered into ten clusters comprising nodes of the samemodel. Each cluster (including the trivial cluster consisting of onlyone node) will require a model load and unload during the subsequentevaluation stage. Accordingly, if the ordered list depicted in FIG. 11 Awere used to evaluate the nodes and cache the evaluations fortessellation, the Rail model would need to be loaded and unloaded once(to evaluate nodes in cluster #1), the Human model would need to beloaded and unloaded four times (to evaluate nodes in clusters #2, #5, #7and #9), the Moon model would need to be loaded and unloaded once (toevaluate nodes in cluster #3), the Targeter model would need to beloaded and unloaded twice (to evaluate nodes in clusters #4 and #6), andCup would need to be loaded and unloaded once (to evaluate nodes incluster #8), and the Phone would need to be loaded and unloaded once (toevaluate nodes in cluster #10).

Since it is quite expensive (from a computing time and memory usageperspective) to load and unload a model, embodiments of the presentinvention attempt to reorder the nodes in the rolled out linear list ofordered nodes to minimize the number of clusters of nodes belonging tothe same model, while satisfying the invariant condition. Optimally,each model should be loaded and unloaded only once, i.e., the orderedlist should comprise only a single cluster for a model; however, thismay not be possible due to the nature of inter-model dependenciespresent in the scene. Accordingly, the goal of the reordering is to haveas few clusters of “same-model-nodes” (i.e. nodes belonging to the samemodel, indicated by the first letter of node name in our FIG. 11A) inthe reordering process as possible. The optimization process isdescribed below.

(2) Cluster Nodes Leftwards

The reordering is initiated by moving each node to the left in order to:

-   RULE #1: If possible, merge with the leftmost legal cluster of the    same model (thus potentially reducing the total number of clusters),    otherwise-   RULE #2: If no such merge is possible, move the node as far left as    it can legally move, so that other nodes that may depend on the node    itself are freer to merge with other clusters.

In applying these two rules, a move is legal for a particular node aslong as the move does not result in the particular node being moved past(to the left of) a node on which the particular node depends (denoted byarrows in the Figs.). Accordingly, a move is illegal if it results in aparticular node being moved past (to the left of) a node upon which itdepends, as that would violate the invariant condition and thusinvalidate the subsequent caching process. The clustering process startsat the beginning of the current linearization, considering each node inturn, and reordering the linearization in place. We will now follow thisprocess through once, showing interesting intermediate orderings of thelinear list. As previously stated, the arrows in the Figs. represent thedependencies depicted in the proxy graph depicted in FIG. 6.

Using the ordered list depicted in FIG. 11A as the starting point, thereordering begins with node RRO in FIG. 11A, which cannot be movedfurther left since it is already as far left as it can go. Likewise,neither rule can be applied to node HRI, because it depends on RRO. Whennode MMO is considered, since there are no other “M” clusters (i.e.,clusters of nodes belonging to the Moon model) to the left of node MMO,Rule #1 does not apply, but Rule #2 can be applied, moving node MMO pastboth nodes HRI and RRO. The same analysis applies to node TTI, whichmoves left until it hits node MMO, upon which it depends. Thisreordering of the nodes produces the ordered list of nodes depicted inFIG. 11B.

As a result of the reordering, it can be seen that HWI can merge withits left neighbor HRI. The node HNO can also merge with its leftneighbor HWI. Accordingly, nodes HRI, HWI, and HNO of the human modelare clustered to form a single cluster #4. This reduces the total numberof clusters from ten in FIG. 11A to nine in FIG. 11B.

Proceeding to node TPI, neither Rule #1 or Rule #2 can be applied to it.Node TOO can be merged with TPI using Rule #1, without actually movingit. Node HHI cannot be moved or merged because it depends on the nodedirectly to its left (node TOO). Node HLO can be moved to the left byapplying Rule #2 until it can merge with a cluster of nodes of the Humanmodel (i.e., cluster #4 depicted in FIG. 11B) to form a larger clusterof nodes. Node CHI can also be moved to the left by applying Rule #2.

After the reordering described above, the ordered list in FIG. 11B istransformed to the ordered list depicted in FIG. 11C. As can be seen,the number of clusters of nodes of the same model has been reduced fromnine in FIG. 11B to eight in FIG. 11C.

Now, Rule #1 can be applied to node HWO to move it left until is mergeswith cluster #4 depicted in FIG. 11C. Rule #2 can be applied to nodePCI, moving it as left as possible, creating a new linearization asdepicted in FIG. 11D. Comparing the linear ordered list depicted in FIG.11D to the linear ordered list depicted in FIG. 11A, the number of timesthat the Human model has to be loaded and unloaded has been reduced fromfour to two, which is the minimum achievable, given the dependencies ofthe scene. The Targeter model still has to be loaded twice, andrepeating the clustering will not improve the ordering further. Next, a“cluster nodes rightwards” technique is applied as described below.

(3) Cluster Nodes Rightwards

An attempt is made to further improve the clustering by merging clusterswith other clusters to the right of their current positions in thelinearization. To do so, the ordered list in FIG. 11D (resulting fromthe clustering nodes leftwards) is used as the starting point andclusters are moved to the right, according to the following two rules:

-   Rule #3: If possible, merge with the rightmost legal cluster of the    same model (thus potentially reducing the total number of clusters),    otherwise-   Rule #4: If no such merge is possible, move the node as far right as    it can legally move, so that other nodes that may depend on the node    itself are freer to merge with other clusters.

Applying this algorithm to the linearization produced by the last step(i.e., the linear ordered list depicted in FIG. 11D), cluster #8 (HHI)and cluster #7 (TOO, TPI) cannot be moved due to their dependencies.Rule #4 can be applied to both clusters #6 (CHI) and #5 (PCI), to givethe linear ordered list configuration depicted in FIG. 11E.

In FIG. 11E, the large cluster #4 of “H” nodes cannot move because TPI,just to its right, depends on one of its nodes, which means that movingpast it would violate the invariant condition. However, Rule #3 can beapplied to cluster #2 (TTI) depicted in FIG. 11E, merging it withcluster #5 (TPI, TOO). Finally, Rule #4 can be applied to cluster #1(MMO) depicted in FIG. 11E, and it can be moved as far right as it cango, resulting in a final ordering as depicted in FIG. 11F. Therightwards-clustering has resulted in merging one more cluster.

(4) Repeat Until Merging Ceases

As seen above, the linearization improves when, as a result of applyingrules in steps 2 and 3, clusters are merged together. Therefore, sinceno action will be taken to split clusters, the termination criterion forthe optimization is when both steps 2 and step 3 fail to merge any moreclusters. It can be verified that a further application of the two stepsto the current linearization depicted in FIG. 11F will only be able toapply steps 2 and 4, which do not affect the number of clusters. Thus,it can be verified that the linear ordered list depicted in FIG. 11Frepresents the optimized list.

The model list of references for the linear list in FIG. 11F is: Rail(to evaluate cluster #1), Human (to evaluate cluster #2), Moon (toevaluate cluster #3), Targeter (to evaluate cluster #4), Human (toevaluate cluster #5), Cup (to evaluate cluster #6), and Phone (toevaluate cluster #7). Accordingly, the Human model must be loaded twiceand the other models must be loaded only once during the evaluation ofthe nodes (inputs and outputs) of the models.

Several iterations of steps 2, 3and 4 may be needed in other embodimentsanalyzing more complicated (realistic) scenes in the productionenvironment before merging is completed (i.e., before an optimizedordered list is arrived at). With the appropriate data structures, eachiteration can be accomplished in order N*F time, where N is the numberof nodes (or proxies) in the proxy connectivity graph and F is themaximum number of direct dependencies any particular node or proxy has.Since F is generally a small constant and does not scale with thecomplexity of the scene, the algorithms therefore scale roughly linearlywith scene complexity, which is the best complexity achievable by anyrenderer. Accordingly, the optimizer does not itself impact overallrendering performance.

Hierarchical Clustering

In the optimization algorithm described above, the only clusteringmetric that is used is the model to which a node belongs. Embodiments ofthe present invention also support the mechanism of hierarchicallygrouping models (one model becomes a child of another model), both toaid in organization of complex scenes, and to facilitate sharing(through inheritance, in which a child inherits from its parent) ofvarious properties that affect the appearance of geometric primitives.

In order to evaluate any model (for caching or final tessellation) thatinherits properties from its ancestors, both the model and all of itsancestors must be simultaneously loaded in a computer's memory. Thismakes it efficient to attempt to consider all of a model's child modelsin succession, due to which a loaded model does not need to be unloadedin order to load some other model's parent.

An elegant generalization of the algorithm presented above enablesoptimization of linearizations to accommodate model hierarchies. Theabove algorithm considers clustering based only on the predicate ofproxy (node) membership in individual models. Let us instead considerallowing a hierarchical sequence of predicates, in which a set ofclusters built by an iteration of the above algorithm using predicate“n” is used as the atomic unit for clustering in iteration “n+1”, whichwill use predicate “n+1”.

This generalization can be used by iterating the optimization algorithmover linit input predicates. For each predicate i, Rule #1 from above ismodified as (similarly for Rule #3):

-   Rule #1(mod): If possible, merge with the leftmost legal cluster of    the same cluster(i) (thus potentially reducing the total number of    clusters), otherwise-   Rule #2(mod): If no such merge is possible, move the cluster as far    left as it can legally move, so that other clusters that may depend    on the cluster itself are freer to merge with other clusters.    Wherein cluster(0) is the zero'th predicate, corresponding to the    model to which the proxies belong, and cluster(i>0) corresponds to    the i'th ancestor of the model.

When an iteration of the optimizing algorithm has been completed foreach predicate in sequence, an optimal ordering is produced for thecaching process.

Interval Rendering

In an embodiment of the present invention, the linear list optimizationalgorithm described above may be extended to take advantage of coherencein multi-frame-rendering. In a typical animation consisting of dozens tohundreds of frames (e.g., 24 rendered frames per second of animation),there is much that is the same or only slightly different from one frameto the next. Inter-model dependencies due to dataflow connections arefrequently coherent over a significant frame range, and by computing asingle optimized linearization that can be used by the tessellator overmultiple sequential frames, the number of times that each model must tobe loaded can be reduced. In most instances, this speeds up thetessellation of “n” frames simultaneously by nearly a factor of “n”.

The algorithm may be configured to enable it to consider proxy graphsthat do not represent a single instant in time, but-rather all thedependencies active over an interval of time. Proxy graphs may furtherbe characterized as a consistent proxy graph, one that can be linearizedinto an ordering that obeys the invariant condition (and thus can beused to accurately evaluate and cache the node proxies), and aninconsistent proxy graph, one that cannot be linearized into an orderingthat obeys the invariant condition because it contains a cycle. Cyclesare easy to detect in the proxy graph (or any graph), and thus enableembodiments of the present invention an efficient means of determiningwhether a given proxy graph is consistent.

Given these extensions to proxy graphs, the goal is to decompose a givenanimation and frame range over the animation into the smallest number ofintervals possible such that:

-   1. The sequential intervals pack tightly (i.e. the union of all the    intervals completely covers the input frame range)-   2. Each interval corresponds to a consistent proxy graph defined    over the interval, which means a single optimized linearization of    the graph can be obtained to use across all frames in the interval.

Making use only of the first step in the optimizer algorithm (producinga valid linearization) to evaluate whether a given proxy graph isconsistent, processing proceeds as follows:

(1) First, calculate, over the frame-range, all the times at which anydataflow relationship changes. The boundaries in time of consistentintervals will always lie on such frames. These frames are then placedinto a change sequence.

(2) Beginning with the entire frame range as the (single) interval,perform binary subdivision using the change sequence until each intervalis consistent. Each test of an interval requires creating the proxygraph for the interval, and verifying whether the proxy graph isconsistent.

Once the intervals have been discovered, the optimizer is executed oncefor the proxy graph of each interval, and the caching phase can evaluateand cache each model over the entire interval without reloading.

Although specific embodiments of the invention have been described,various modifications, alterations, alternative constructions, andequivalents are also encompassed within the scope of the invention. Thedescribed invention is not restricted to operation within certainspecific data processing environments, but is free to operate within aplurality of data processing environments. Additionally, although thepresent invention has been described using a particular series oftransactions and steps, it should be apparent to those skilled in theart that the scope of the present invention is not limited to thedescribed series of transactions and steps.

Further, while the present invention has been described using aparticular combination of hardware and software (e.g., software codemodules, instructions), it should be recognized that other combinationsof hardware and software are also within the scope of the presentinvention. The present invention may be implemented only in hardware, oronly in software, or using combinations thereof. For example, theprocessing performed by the present invention, as described above, maybe implemented in hardware chips, graphics boards or accelerators, etc.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that additions, subtractions, deletions, and other modificationsand changes may be made thereunto without departing from the broaderspirit and scope of the invention as set forth in the claims.

1. A computer-implemented method of facilitating rendering of a scenecomprising a plurality of models, the method comprising: determining anordered list of model references using a first representation, the firstrepresentation comprising information identifying one or more inputs andoutputs of the models in the plurality of models and identifyingdependencies between the one or more inputs and outputs, each modelreference identifying a model from the plurality of models, whereindetermining the ordered list of model references comprises: generating,based upon the first representation, an ordered list of inputs andoutputs represented by the first representation; determining one or moreclusters in the ordered list of inputs and outputs, wherein each clustercomprises one or more inputs or outputs of the same model that arecontiguous to each other in the ordered list of inputs and outputs;generating a final ordered list of inputs and outputs by reordering theinputs and outputs in the ordered list of inputs and outputs to reducethe number of clusters comprising one or more inputs or outputs of amodel; ordering the clusters in the final ordered list of inputs andoutputs; and determining the ordered list of model references based uponthe ordering of the clusters; and processing the model references inorder of their position in the ordered list, the processing for eachmodel reference comprising: loading a model corresponding to the modelreference in computer memory, wherein the model is loaded a number oftimes equal to the number of clusters of the model, the inputs andoutputs of one of the clusters being evaluated each time the model isloaded; generating a result using the loaded model; and storing thegenerated result; and rendering the scene using the stored results,wherein the rendering is performed without loading all of the pluralityof models concurrently in the computer memory.
 2. The method of claim 1,wherein the first representation is a data structure comprising aplurality of nodes and directed links between the nodes, wherein thenodes represent the inputs and the outputs of the plurality of modelsand the directed links represent the dependencies between the inputs andthe outputs.
 3. The method of claim 1, wherein rendering the scenecomprises: generating a tessellated representation for each model in theplurality of models using the stored results, wherein the tessellatedrepresentation for each model is generated without loading the pluralityof models concurrently in the computer memory; and rendering the sceneusing the tessellated representations of the plurality of model.
 4. Themethod of claim 1, wherein generating a result using the loaded modelfor a model reference comprises using a previously stored result togenerate the result.
 5. A computer-readable medium encoded with acomputer program for facilitating rendering of a scene comprising aplurality of models, the computer program comprising: code fordetermining an ordered list of model references using a firstrepresentation, the first representation comprising informationidentifying one or more inputs and outputs of the models in theplurality of models and identifying dependencies between the one or moreinputs and outputs, each model reference identifying a model from theplurality of models, wherein the code for determining the ordered listof model references comprises: code for generating, based upon the firstrepresentation, an ordered list of inputs and outputs represented by thefirst representation; code for determining one or more clusters in theordered list of inputs and outputs, wherein each cluster comprises oneor more inputs or outputs of the same model that are contiguous to eachother in the ordered list of inputs and outputs; code for generating afinal ordered list of inputs and outputs by reordering the inputs andoutputs in the ordered list of inputs and outputs to reduce the numberof clusters comprising one or more inputs or outputs of a model; andcode for ordering the clusters in the final ordered list of inputs andoutputs; and code for determining the ordered list of model referencesbased upon the ordering of the clusters; and code for processing themodel references in order of their position in the ordered list, theprocessing for each model reference comprising: loading a modelcorresponding to the model reference in computer memory, wherein themodel is loaded a number of times equal to the number of clusters of themodel, the inputs and outputs of one of the clusters being evaluatedeach time the model is loaded; generating a result using the loadedmodel; and storing the generated result; and code for rendering thescene using the stored results, wherein the rendering is performedwithout loading all of the plurality of models concurrently in thecomputer memory.
 6. The computer-readable medium of claim 5, wherein thefirst representation is a data structure comprising a plurality of nodesand directed links between the nodes, wherein the nodes represent theinputs and the outputs of the plurality of models and the directed linksrepresent the dependencies between the inputs and the outputs.
 7. Thecomputer-readable medium of claim 5, wherein the code for rendering thescene comprises: code for generating a tessellated representation foreach model in the plurality of models using the stored results, whereinthe tessellated representation for each model is generated withoutloading the plurality of models concurrently in the computer memory; andcode for rendering the scene using the tessellated representations ofthe plurality of models.
 8. The computer-readable medium of claim 5,wherein generating a result using the loaded model for a model referencecomprises using a previously stored result to generate the result.
 9. Asystem for facilitating rendering of a scene comprising a plurality ofmodels, the system comprising: a data processing system; a renderercoupled to the data processing system; wherein the data processingsystem is configured to: determine an ordered list of model referencesusing a first representation, the first representation comprisinginformation identifying one or more inputs and outputs of the models inthe plurality of models and identifying dependencies between the one ormore inputs and outputs, each model reference identifying a model fromthe plurality of models, wherein determining the ordered list of modelreferences comprises: generating, based upon the first representation,an ordered list of inputs and outputs represented by the firstrepresentation; determining one or more clusters in the ordered list ofinputs and outputs, wherein each cluster comprises one or more inputs oroutputs of the same model that are contiguous to each other in theordered list of inputs and outputs; generating a final ordered list ofinputs and outputs by reordering the inputs and outputs in the orderedlist of inputs and outputs to reduce the number of clusters comprisingone or more inputs or outputs of a model; and ordering the clusters inthe final ordered list of inputs and outputs; and determining theordered list of model references based upon the ordering of theclusters; and process the model references in order of their position inthe ordered list, the processing for each model reference comprising:loading a model corresponding to the model reference in computer memoryof the data processing system, wherein the model is loaded a number oftimes equal to the number of clusters of the model, the inputs andoutputs of one of the clusters being evaluated each time the model isloaded; generating a result using the loaded model; and storing thegenerated result; and wherein the renderer is configured to render thescene using the stored results, wherein the rendering is performedwithout loading all of the plurality of models concurrently in computermemory of the renderer.
 10. The system of claim 9, wherein the firstrepresentation is a data structure comprising a plurality of nodes anddirected links between the nodes, wherein the nodes represent the inputsand the outputs of the plurality of models and the directed linksrepresent the dependencies between the inputs and the outputs.
 11. Thesystem of claim 9, further comprising a tessellator configured togenerate a tessellated representation for each model in the plurality ofmodels using the stored results, wherein the tessellated representationfor each model is generated without loading the plurality of modelsconcurrently in the computer memory; and wherein the renderer isconfigured to render the scene using the tessellated representations ofthe plurality of models.
 12. The system of claim 9, wherein the dataprocessing system is configured to generate a result for a modelreference using a previously stored result.